{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "provenance": [],
      "collapsed_sections": [
        "jv1SzsYLVl-q"
      ]
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "source": [
        "# 零.前言\n",
        "\n"
      ],
      "metadata": {
        "id": "1E0WjcfcQJpL"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "#### 使用的模型：din\n",
        "#### 模型功能：预测用户是否想要购买目标商品y\n",
        "#### 模型特点：该模型一般用于分析用户历史行为与预测目标之前的关系\n",
        "#### 数据集：用户的id，亚马逊的用户购买商品的记录，以及要预测的目标商品（注：一般更好的数据集会带有用户的个人信息，也可以作为模型输入进行分析，但是作为一个模板，不搞这么复制）\n",
        "#### 模型输入，用户的历史购买商品类别序列\n",
        "#### 模型输出，用户想购买商品的概率\n",
        "#### 模型亮点：引入了注意力机制，会去分析商品与推荐商品之间的注意力系数\n",
        "\n",
        "#### 模型论文链接:https://arxiv.org/pdf/1706.06978.pdf\n"
      ],
      "metadata": {
        "id": "jv1SzsYLVl-q"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "# 一.导入包"
      ],
      "metadata": {
        "id": "gP1Tsp-sVSm6"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "#### 1.可能会显示有些包没有，建立一个新的代码快，输入[!pip insatll xxxx]安装没有的包\n",
        "#### 2.如果你用想google colab来跑的话，弄个谷歌账号，注册google drive，在对应的位置放上数据集，如果在本地跑的话，下面的注释4的代码干掉，"
      ],
      "metadata": {
        "id": "401IuBgnVpL6"
      }
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "zcX_td12DRX0",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "11e21084-a55a-4a16-cc82-bda04a9b6933"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Drive already mounted at /content/gdrive; to attempt to forcibly remount, call drive.mount(\"/content/gdrive\", force_remount=True).\n"
          ]
        }
      ],
      "source": [
        "'''\n",
        "-*- coding: utf-8 -*-\n",
        "@File  : din.py\n",
        "'''\n",
        "# 1.python自定义包\n",
        "import os\n",
        "import pandas as pd\n",
        "import numpy as np\n",
        "# 2.pytorch相关包\n",
        "import torch\n",
        "import torch.nn as nn\n",
        "import torch.optim as optim\n",
        "import torch.utils.data as Data\n",
        "import torch.nn.functional as F\n",
        "# 3.sklearn相关包\n",
        "from sklearn.preprocessing import LabelEncoder, OrdinalEncoder, KBinsDiscretizer\n",
        "from sklearn.model_selection import train_test_split\n",
        "from sklearn.metrics import roc_auc_score\n",
        "from sklearn import metrics\n",
        "# 4.链接google_drive获取数据集，注：在google相应的位置放入数据文件\n",
        "from google.colab import drive\n",
        "drive.mount('/content/gdrive')"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "!ls"
      ],
      "metadata": {
        "id": "kjpP23Sxw-y6",
        "outputId": "ebdc3817-0eaa-4300-c2c5-043a888f7ca6",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "sample_data\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# 二.导入数据"
      ],
      "metadata": {
        "id": "aQuKNlz6VbAS"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "#### 1.数据集读取，这里是我google drive存放数据集的路径，在Colab Notebooks文件夹下，创建model文件夹放入amazon...的txt数据集，如果放在本地跑的话read_csv()里面的内容改成你的本地路径\n",
        "#### 2.数据集的内容-label：实际上用户有没有购买商品，userid：用户唯一id，itemid：商品唯一id，cateid：商品类别，hist_item_list：购买的商品id序列，hist_cate_list：购买的商品类别序列\n",
        "\n"
      ],
      "metadata": {
        "id": "fbrZFfhQWZma"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "data = pd.read_csv('/content/gdrive/MyDrive/Colab Notebooks/model/amazon-books-100k.txt')"
      ],
      "metadata": {
        "id": "s2f83q8WDhLW"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "data"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 424
        },
        "id": "5iwvt48mHy_M",
        "outputId": "db4bc5c4-c4f0-41da-ae2d-f22de6f9c3d7"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "       label          userID      itemID                cateID  \\\n",
              "0          0   AZPJ9LUT0FEPY  B00AMNNTIA  Literature & Fiction   \n",
              "1          1   AZPJ9LUT0FEPY  0800731603                 Books   \n",
              "2          0  A2NRV79GKAU726  B003NNV10O               Russian   \n",
              "3          1  A2NRV79GKAU726  B000UWJ91O                 Books   \n",
              "4          0  A2GEQVDX2LL4V3  0321334094                 Books   \n",
              "...      ...             ...         ...                   ...   \n",
              "89994      0  A3CV7NJJC20JTB  098488789X                 Books   \n",
              "89995      1  A3CV7NJJC20JTB  0307381277                 Books   \n",
              "89996      0  A208PSIK2APSKN  0957496184                 Books   \n",
              "89997      1  A208PSIK2APSKN  1480198854                 Books   \n",
              "89998      0  A1GRLKG8JA19OA  B0095VGR4I  Literature & Fiction   \n",
              "\n",
              "                                          hist_item_list  \\\n",
              "0      0307744434|0062248391|0470530707|0978924622|15...   \n",
              "1      0307744434|0062248391|0470530707|0978924622|15...   \n",
              "2      0814472869|0071462074|1583942300|0812538366|B0...   \n",
              "3      0814472869|0071462074|1583942300|0812538366|B0...   \n",
              "4            0743596870|0374280991|1439140634|0976475731   \n",
              "...                                                  ...   \n",
              "89994        034545197X|0765326396|1605420832|1451648448   \n",
              "89995        034545197X|0765326396|1605420832|1451648448   \n",
              "89996  0515140791|147674355X|B0055ECOUA|B007JE1B1C|B0...   \n",
              "89997  0515140791|147674355X|B0055ECOUA|B007JE1B1C|B0...   \n",
              "89998        031612091X|0399163832|1442358238|1118017447   \n",
              "\n",
              "                                          hist_cate_list  \n",
              "0                          Books|Books|Books|Books|Books  \n",
              "1                          Books|Books|Books|Books|Books  \n",
              "2             Books|Books|Books|Books|Baking|Books|Books  \n",
              "3             Books|Books|Books|Books|Baking|Books|Books  \n",
              "4                                Books|Books|Books|Books  \n",
              "...                                                  ...  \n",
              "89994                            Books|Books|Books|Books  \n",
              "89995                            Books|Books|Books|Books  \n",
              "89996  Books|Books|Bibles|Literature & Fiction|Litera...  \n",
              "89997  Books|Books|Bibles|Literature & Fiction|Litera...  \n",
              "89998                            Books|Books|Books|Books  \n",
              "\n",
              "[89999 rows x 6 columns]"
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-7dc2c497-501f-4943-9def-76735dfb4dca\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>label</th>\n",
              "      <th>userID</th>\n",
              "      <th>itemID</th>\n",
              "      <th>cateID</th>\n",
              "      <th>hist_item_list</th>\n",
              "      <th>hist_cate_list</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>0</td>\n",
              "      <td>AZPJ9LUT0FEPY</td>\n",
              "      <td>B00AMNNTIA</td>\n",
              "      <td>Literature &amp; Fiction</td>\n",
              "      <td>0307744434|0062248391|0470530707|0978924622|15...</td>\n",
              "      <td>Books|Books|Books|Books|Books</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>1</td>\n",
              "      <td>AZPJ9LUT0FEPY</td>\n",
              "      <td>0800731603</td>\n",
              "      <td>Books</td>\n",
              "      <td>0307744434|0062248391|0470530707|0978924622|15...</td>\n",
              "      <td>Books|Books|Books|Books|Books</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>0</td>\n",
              "      <td>A2NRV79GKAU726</td>\n",
              "      <td>B003NNV10O</td>\n",
              "      <td>Russian</td>\n",
              "      <td>0814472869|0071462074|1583942300|0812538366|B0...</td>\n",
              "      <td>Books|Books|Books|Books|Baking|Books|Books</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>1</td>\n",
              "      <td>A2NRV79GKAU726</td>\n",
              "      <td>B000UWJ91O</td>\n",
              "      <td>Books</td>\n",
              "      <td>0814472869|0071462074|1583942300|0812538366|B0...</td>\n",
              "      <td>Books|Books|Books|Books|Baking|Books|Books</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>0</td>\n",
              "      <td>A2GEQVDX2LL4V3</td>\n",
              "      <td>0321334094</td>\n",
              "      <td>Books</td>\n",
              "      <td>0743596870|0374280991|1439140634|0976475731</td>\n",
              "      <td>Books|Books|Books|Books</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>...</th>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>89994</th>\n",
              "      <td>0</td>\n",
              "      <td>A3CV7NJJC20JTB</td>\n",
              "      <td>098488789X</td>\n",
              "      <td>Books</td>\n",
              "      <td>034545197X|0765326396|1605420832|1451648448</td>\n",
              "      <td>Books|Books|Books|Books</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>89995</th>\n",
              "      <td>1</td>\n",
              "      <td>A3CV7NJJC20JTB</td>\n",
              "      <td>0307381277</td>\n",
              "      <td>Books</td>\n",
              "      <td>034545197X|0765326396|1605420832|1451648448</td>\n",
              "      <td>Books|Books|Books|Books</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>89996</th>\n",
              "      <td>0</td>\n",
              "      <td>A208PSIK2APSKN</td>\n",
              "      <td>0957496184</td>\n",
              "      <td>Books</td>\n",
              "      <td>0515140791|147674355X|B0055ECOUA|B007JE1B1C|B0...</td>\n",
              "      <td>Books|Books|Bibles|Literature &amp; Fiction|Litera...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>89997</th>\n",
              "      <td>1</td>\n",
              "      <td>A208PSIK2APSKN</td>\n",
              "      <td>1480198854</td>\n",
              "      <td>Books</td>\n",
              "      <td>0515140791|147674355X|B0055ECOUA|B007JE1B1C|B0...</td>\n",
              "      <td>Books|Books|Bibles|Literature &amp; Fiction|Litera...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>89998</th>\n",
              "      <td>0</td>\n",
              "      <td>A1GRLKG8JA19OA</td>\n",
              "      <td>B0095VGR4I</td>\n",
              "      <td>Literature &amp; Fiction</td>\n",
              "      <td>031612091X|0399163832|1442358238|1118017447</td>\n",
              "      <td>Books|Books|Books|Books</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "<p>89999 rows × 6 columns</p>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-7dc2c497-501f-4943-9def-76735dfb4dca')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "      \n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-7dc2c497-501f-4943-9def-76735dfb4dca button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-7dc2c497-501f-4943-9def-76735dfb4dca');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n",
              "  "
            ]
          },
          "metadata": {},
          "execution_count": 56
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# 三.数据处理"
      ],
      "metadata": {
        "id": "B2CgVmKlYqpC"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "#### 1.由于该数据集只有10w条，很多商品id只出现了一次，故编码的时候是以类别作为编码和预测的targe\n",
        "#### 2.如果你要用学生做的试题序列作为训练集，且这些试题被不同的学生来回做过，可以用试题作为唯一的编码\n",
        "#### 3.这里数据处理的目的是形成学生答题序列，把文本数据转化为唯一的数值编码，作为模型的输入，用于预测的目标试题\n"
      ],
      "metadata": {
        "id": "UCvAo3i7VghD"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "def AmazonBookPreprocess(dataframe, seq_len=40):\n",
        "    \"\"\"\n",
        "    数据集处理\n",
        "    :param dataframe: 未处理的数据集\n",
        "    :param seq_len: 数据序列长度\n",
        "    :return data: 处理好的数据集\n",
        "    \"\"\"\n",
        "    # 1.按'|'切割，用户历史购买数据，获取item的序列和类别的序列\n",
        "    data = dataframe.copy()\n",
        "    data['hist_item_list'] = dataframe.apply(lambda x: x['hist_item_list'].split('|'), axis=1)\n",
        "    data['hist_cate_list'] = dataframe.apply(lambda x: x['hist_cate_list'].split('|'), axis=1)\n",
        "\n",
        "    # 2.获取cate的所有种类，为每个类别设置一个唯一的编码\n",
        "    cate_list = list(data['cateID'])\n",
        "    _ = [cate_list.extend(i) for i in data['hist_cate_list'].values]\n",
        "    # 3.将编码去重\n",
        "    cate_set = set(cate_list + ['0'])  # 用 '0' 作为padding的类别\n",
        "\n",
        "    # 4.截取用户行为的长度,也就是截取hist_cate_list的长度，生成对应的列名\n",
        "    cols = ['hist_cate_{}'.format(i) for i in range(seq_len)]\n",
        "\n",
        "    # 5.截取前40个历史行为，如果历史行为不足40个则填充0\n",
        "    def trim_cate_list(x):\n",
        "        if len(x) > seq_len:\n",
        "            # 5.1历史行为大于40, 截取后40个行为\n",
        "            return pd.Series(x[-seq_len:], index=cols)\n",
        "        else:\n",
        "            # 5.2历史行为不足40, padding到40个行为\n",
        "            pad_len = seq_len - len(x)\n",
        "            x = x + ['0'] * pad_len\n",
        "            return pd.Series(x, index=cols)\n",
        "\n",
        "    # 6.预测目标为试题的类别\n",
        "    labels = data['label']\n",
        "    data = data['hist_cate_list'].apply(trim_cate_list).join(data['cateID'])\n",
        "\n",
        "    # 7.生成类别对应序号的编码器，如book->1,Russian->2这样\n",
        "    cate_encoder = LabelEncoder().fit(list(cate_set))\n",
        "    # 8.这里分为两步，第一步为把类别转化为数值，第二部为拼接上label\n",
        "    data = data.apply(cate_encoder.transform).join(labels)\n",
        "    return data"
      ],
      "metadata": {
        "id": "GoqOuuLCXhBn"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# 对数据进行处理\n",
        "cate_encoder = None\n",
        "data = AmazonBookPreprocess(data)"
      ],
      "metadata": {
        "id": "Cr7g0E-HDhZ_"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# 形成历史购买序列和label的数据集\n",
        "data"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 488
        },
        "id": "rLXFKB8uI-bz",
        "outputId": "bb8d18f3-fa53-488b-f311-25dd1f4f94b9"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "       hist_cate_0  hist_cate_1  hist_cate_2  hist_cate_3  hist_cate_4  \\\n",
              "0              138          138          138          138          138   \n",
              "1              138          138          138          138          138   \n",
              "2              138          138          138          138           95   \n",
              "3              138          138          138          138           95   \n",
              "4              138          138          138          138            0   \n",
              "...            ...          ...          ...          ...          ...   \n",
              "89994          138          138          138          138            0   \n",
              "89995          138          138          138          138            0   \n",
              "89996          138          138          115          734          734   \n",
              "89997          138          138          115          734          734   \n",
              "89998          138          138          138          138            0   \n",
              "\n",
              "       hist_cate_5  hist_cate_6  hist_cate_7  hist_cate_8  hist_cate_9  ...  \\\n",
              "0                0            0            0            0            0  ...   \n",
              "1                0            0            0            0            0  ...   \n",
              "2              138          138            0            0            0  ...   \n",
              "3              138          138            0            0            0  ...   \n",
              "4                0            0            0            0            0  ...   \n",
              "...            ...          ...          ...          ...          ...  ...   \n",
              "89994            0            0            0            0            0  ...   \n",
              "89995            0            0            0            0            0  ...   \n",
              "89996            0            0            0            0            0  ...   \n",
              "89997            0            0            0            0            0  ...   \n",
              "89998            0            0            0            0            0  ...   \n",
              "\n",
              "       hist_cate_32  hist_cate_33  hist_cate_34  hist_cate_35  hist_cate_36  \\\n",
              "0                 0             0             0             0             0   \n",
              "1                 0             0             0             0             0   \n",
              "2                 0             0             0             0             0   \n",
              "3                 0             0             0             0             0   \n",
              "4                 0             0             0             0             0   \n",
              "...             ...           ...           ...           ...           ...   \n",
              "89994             0             0             0             0             0   \n",
              "89995             0             0             0             0             0   \n",
              "89996             0             0             0             0             0   \n",
              "89997             0             0             0             0             0   \n",
              "89998             0             0             0             0             0   \n",
              "\n",
              "       hist_cate_37  hist_cate_38  hist_cate_39  cateID  label  \n",
              "0                 0             0             0     734      0  \n",
              "1                 0             0             0     138      1  \n",
              "2                 0             0             0    1071      0  \n",
              "3                 0             0             0     138      1  \n",
              "4                 0             0             0     138      0  \n",
              "...             ...           ...           ...     ...    ...  \n",
              "89994             0             0             0     138      0  \n",
              "89995             0             0             0     138      1  \n",
              "89996             0             0             0     138      0  \n",
              "89997             0             0             0     138      1  \n",
              "89998             0             0             0     734      0  \n",
              "\n",
              "[89999 rows x 42 columns]"
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-9e85d8ba-e9be-4698-a24e-68d77dc0a028\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>hist_cate_0</th>\n",
              "      <th>hist_cate_1</th>\n",
              "      <th>hist_cate_2</th>\n",
              "      <th>hist_cate_3</th>\n",
              "      <th>hist_cate_4</th>\n",
              "      <th>hist_cate_5</th>\n",
              "      <th>hist_cate_6</th>\n",
              "      <th>hist_cate_7</th>\n",
              "      <th>hist_cate_8</th>\n",
              "      <th>hist_cate_9</th>\n",
              "      <th>...</th>\n",
              "      <th>hist_cate_32</th>\n",
              "      <th>hist_cate_33</th>\n",
              "      <th>hist_cate_34</th>\n",
              "      <th>hist_cate_35</th>\n",
              "      <th>hist_cate_36</th>\n",
              "      <th>hist_cate_37</th>\n",
              "      <th>hist_cate_38</th>\n",
              "      <th>hist_cate_39</th>\n",
              "      <th>cateID</th>\n",
              "      <th>label</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>138</td>\n",
              "      <td>138</td>\n",
              "      <td>138</td>\n",
              "      <td>138</td>\n",
              "      <td>138</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>...</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>734</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>138</td>\n",
              "      <td>138</td>\n",
              "      <td>138</td>\n",
              "      <td>138</td>\n",
              "      <td>138</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>...</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>138</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>138</td>\n",
              "      <td>138</td>\n",
              "      <td>138</td>\n",
              "      <td>138</td>\n",
              "      <td>95</td>\n",
              "      <td>138</td>\n",
              "      <td>138</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>...</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1071</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>138</td>\n",
              "      <td>138</td>\n",
              "      <td>138</td>\n",
              "      <td>138</td>\n",
              "      <td>95</td>\n",
              "      <td>138</td>\n",
              "      <td>138</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>...</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>138</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>138</td>\n",
              "      <td>138</td>\n",
              "      <td>138</td>\n",
              "      <td>138</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>...</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>138</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>...</th>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>89994</th>\n",
              "      <td>138</td>\n",
              "      <td>138</td>\n",
              "      <td>138</td>\n",
              "      <td>138</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>...</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>138</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>89995</th>\n",
              "      <td>138</td>\n",
              "      <td>138</td>\n",
              "      <td>138</td>\n",
              "      <td>138</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>...</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>138</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>89996</th>\n",
              "      <td>138</td>\n",
              "      <td>138</td>\n",
              "      <td>115</td>\n",
              "      <td>734</td>\n",
              "      <td>734</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>...</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>138</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>89997</th>\n",
              "      <td>138</td>\n",
              "      <td>138</td>\n",
              "      <td>115</td>\n",
              "      <td>734</td>\n",
              "      <td>734</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>...</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>138</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>89998</th>\n",
              "      <td>138</td>\n",
              "      <td>138</td>\n",
              "      <td>138</td>\n",
              "      <td>138</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>...</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>734</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "<p>89999 rows × 42 columns</p>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-9e85d8ba-e9be-4698-a24e-68d77dc0a028')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "      \n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-9e85d8ba-e9be-4698-a24e-68d77dc0a028 button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-9e85d8ba-e9be-4698-a24e-68d77dc0a028');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n",
              "  "
            ]
          },
          "metadata": {},
          "execution_count": 24
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# 查看是否有gpu进行运算，如果没有则使用cpu运算（注：cpu计算很慢很慢，最好开个gpu进行计算）\n",
        "device=torch.device(\"cuda:0\" if torch.cuda.is_available() else \"cpu\")\n",
        "# 计算出现的最大类别编码是多少，目的为统计一共有多少个商品类别\n",
        "fields = data.max().max()"
      ],
      "metadata": {
        "id": "HvSLnGPADhc3"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "# 四.模型定义\n"
      ],
      "metadata": {
        "id": "amVNJEWoY-Y-"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "#### 1.这块代码主要是，定义din模型，模型的内容可以看https://arxiv.org/pdf/1706.06978.pdf 了解din模型是个啥\n",
        "#### 2.pytorch定义的模型主要是看init和forward，\n",
        "#### 3.init的功能是初始化一些变量，和别的类定义一样；\n",
        "#### 4.forward的功能是向前传播，是调用类时直接将参数输入forward中，进行计算；"
      ],
      "metadata": {
        "id": "YM2faH1mZHYa"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "class Dice(nn.Module):\n",
        "    \"\"\"\n",
        "    自定义的dice激活函数，原论文有公式介绍，有点复杂我也没看懂，别的地方用的不多，不介绍了。\n",
        "    \"\"\"\n",
        "    def __init__(self):\n",
        "        super(Dice, self).__init__()\n",
        "        self.alpha = nn.Parameter(torch.zeros((1,)))\n",
        "        self.epsilon = 1e-9\n",
        "    \n",
        "    def forward(self, x):\n",
        "\n",
        "        norm_x = (x - x.mean(dim=0)) / torch.sqrt(x.var(dim=0) + self.epsilon)\n",
        "        p = torch.sigmoid(norm_x)\n",
        "        x = self.alpha * x.mul(1-p) + x.mul(p)\n",
        "    \n",
        "        return x\n",
        "\n"
      ],
      "metadata": {
        "id": "o3voLAKjjbux"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "class ActivationUnit(nn.Module):\n",
        "    \"\"\"\n",
        "    激活函数单元\n",
        "    功能是计算用户购买行为与推荐目标之间的注意力系数，比如说用户虽然用户买了这个东西，但是这个东西实际上和推荐目标之间没啥关系，也不重要，所以要乘以一个小权重\n",
        "    \"\"\"\n",
        "    def __init__(self, embedding_dim, dropout=0.2, fc_dims = [32, 16]):\n",
        "        super(ActivationUnit, self).__init__()\n",
        "        # 1.初始化fc层\n",
        "        fc_layers = []\n",
        "        # 2.输入特征维度\n",
        "        input_dim = embedding_dim*4     \n",
        "        # 3.fc层内容：全连接层（4*embedding,32）—>激活函数->dropout->全连接层（32,16）->.....->全连接层（16,1）\n",
        "        for fc_dim in fc_dims:\n",
        "            fc_layers.append(nn.Linear(input_dim, fc_dim))\n",
        "            fc_layers.append(Dice())\n",
        "            fc_layers.append(nn.Dropout(p = dropout))\n",
        "            input_dim = fc_dim\n",
        "        \n",
        "        fc_layers.append(nn.Linear(input_dim, 1))\n",
        "        # 4.将上面定义的fc层，整合到sequential中\n",
        "        self.fc = nn.Sequential(*fc_layers)\n",
        "    \n",
        "    def forward(self, query, user_behavior):\n",
        "        \"\"\"\n",
        "            :param query:targe目标的embedding ->（输入维度） batch*1*embed \n",
        "            :param user_behavior:行为特征矩阵 ->（输入维度） batch*seq_len*embed\n",
        "            :return out:预测目标与历史行为之间的注意力系数\n",
        "        \"\"\"\n",
        "        # 1.获取用户历史行为序列长度\n",
        "        seq_len = user_behavior.shape[1]\n",
        "        # 2.序列长度*embedding\n",
        "        queries = torch.cat([query] * seq_len, dim=1)\n",
        "        # 3.前面的把四个embedding合并成一个（4*embedding）的向量，\n",
        "        #  第一个向量是目标商品的向量，第二个向量是用户行为的向量，\n",
        "        #  至于第三个和第四个则是他们的相减和相乘（这里猜测是为了添加一点非线性数据用于全连接层，充分训练）\n",
        "        attn_input = torch.cat([queries, user_behavior, queries - user_behavior, \n",
        "                                queries * user_behavior], dim = -1)\n",
        "        out = self.fc(attn_input)\n",
        "        return out"
      ],
      "metadata": {
        "id": "3omTBy-SDhio"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "class AttentionPoolingLayer(nn.Module):\n",
        "    \"\"\"\n",
        "      注意力序列层\n",
        "      功能是计算用户行为与预测目标之间的系数，并将所有的向量进行相加，这里的目的是计算出用户的兴趣的能力向量\n",
        "    \"\"\"\n",
        "    def __init__(self, embedding_dim,  dropout):\n",
        "        super(AttentionPoolingLayer, self).__init__()\n",
        "        self.active_unit = ActivationUnit(embedding_dim = embedding_dim, \n",
        "                                          dropout = dropout)\n",
        "        \n",
        "    def forward(self, query_ad, user_behavior, mask):\n",
        "        \"\"\"\n",
        "          :param query_ad:targe目标x的embedding   -> （输入维度） batch*1*embed\n",
        "          :param user_behavior:行为特征矩阵     -> （输入维度） batch*seq_len*embed\n",
        "          :param mask:被padding为0的行为置为false  -> （输入维度） batch*seq_len*1\n",
        "          :return output:用户行为向量之和，反应用户的爱好\n",
        "        \"\"\"\n",
        "        # 1.计算目标和历史行为之间的相关性\n",
        "        attns = self.active_unit(query_ad, user_behavior)     \n",
        "        # 2.注意力系数乘以行为 \n",
        "        output = user_behavior.mul(attns.mul(mask))\n",
        "        # 3.历史行为向量相加\n",
        "        output = user_behavior.sum(dim=1)\n",
        "        return output\n",
        "    "
      ],
      "metadata": {
        "id": "HCAJZ6PElzJj"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "class DeepInterestNet(nn.Module):\n",
        "    \"\"\"\n",
        "      模型主体\n",
        "      功能是用户最近的历史40个购买物品是xxx时，购买y的概率是多少\n",
        "    \"\"\"\n",
        "\n",
        "    def __init__(self, feature_dim, embed_dim, mlp_dims, dropout):\n",
        "        super(DeepInterestNet, self).__init__()\n",
        "        # 1.特征维度，就是输入的特征有多少个类\n",
        "        self.feature_dim = feature_dim\n",
        "        # 2.embeding层，将特征数值转化为向量\n",
        "        self.embedding = nn.Embedding(feature_dim+1, embed_dim)\n",
        "        # 3.注意力计算层（论文核心）\n",
        "        self.AttentionActivate = AttentionPoolingLayer(embed_dim, dropout)\n",
        "        # 4.定义fc层\n",
        "        fc_layers = []\n",
        "        # 5.该层的输入为历史行为的embedding，和目标的embedding，所以输入维度为2*embedding_dim\n",
        "        #  全连接层（2*embedding,fc_dims[0]）—>激活函数->dropout->全连接层（fc_dims[0],fc_dims[1]）->.....->全连接层（fc_dims[n],1）\n",
        "        input_dim = embed_dim * 2      \n",
        "        for fc_dim in mlp_dims:\n",
        "            fc_layers.append(nn.Linear(input_dim, fc_dim))\n",
        "            fc_layers.append(nn.ReLU())\n",
        "            fc_layers.append(nn.Dropout(p = dropout))\n",
        "            input_dim = fc_dim\n",
        "        fc_layers.append(nn.Linear(input_dim, 1))\n",
        "        # 6.将所有层封装\n",
        "        self.mlp = nn.Sequential(*fc_layers)        \n",
        "    \n",
        "    def forward(self, x):\n",
        "        \"\"\"\n",
        "            x输入(behaviors*40,ads*1) ->（输入维度） batch*(behaviors+ads)\n",
        "            \n",
        "        \"\"\"\n",
        "        # 1.排除掉推荐目标\n",
        "        behaviors_x = x[:,:-1]\n",
        "        # 2.记录之前填充为0的行为位置\n",
        "        mask = (behaviors_x > 0).float().unsqueeze(-1)\n",
        "        # 3.获取推荐的目标\n",
        "        ads_x = x[:,-1]\n",
        "        # 4.对推荐目标进行向量嵌入\n",
        "        query_ad = self.embedding(ads_x).unsqueeze(1)\n",
        "        # 5.对用户行为进行embeding，注意这里的维度为(batch*历史行为长度*embedding长度)\n",
        "        user_behavior = self.embedding(behaviors_x)\n",
        "        # 6.矩阵相乘，将那些行为为空的地方全部写为0\n",
        "        user_behavior = user_behavior.mul(mask)\n",
        "        # 7.将用户行为乘上注意力系数,再把所有行为记录向量相加\n",
        "        user_interest = self.AttentionActivate(query_ad, user_behavior, mask)\n",
        "        # 8.将计算后的用户行为行为记录和推荐的目标进行拼接\n",
        "        concat_input = torch.cat([user_interest, query_ad.squeeze(1)], dim = 1)\n",
        "        # 9.输入用户行为和目标向量，计算预测得分\n",
        "        out = self.mlp(concat_input)\n",
        "        # 10.sigmoid激活函数\n",
        "        out = torch.sigmoid(out.squeeze(1))        \n",
        "        return out"
      ],
      "metadata": {
        "id": "C-NaQvMtjhED"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "# 五.封装训练集，测试集"
      ],
      "metadata": {
        "id": "cI6Ggblnl8h7"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "#### 这里的目的是为了划分训练集，测试集\n",
        "#### 再把数据封装到data_loader里面，方便后面按batch获取数据，训练模型"
      ],
      "metadata": {
        "id": "M8laPI4BmHwU"
      }
    },
    {
      "cell_type": "code",
      "source": [],
      "metadata": {
        "id": "n0cgFdA-Dhl2"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "#模型输入\n",
        "data_X = data.iloc[:,:-1]\n",
        "#模型输出\n",
        "data_y = data.label.values\n",
        "#划分训练集，测试集，验证集\n",
        "tmp_X, test_X, tmp_y, test_y = train_test_split(data_X, data_y, test_size = 0.2, random_state=42, stratify=data_y)\n",
        "train_X, val_X, train_y, val_y = train_test_split(tmp_X, tmp_y, test_size = 0.25, random_state=42, stratify=tmp_y)\n",
        "dis_test_x = test_X\n",
        "dis_test_y = test_y\n",
        "# numpy转化为torch\n",
        "train_X = torch.from_numpy(train_X.values).long()\n",
        "val_X = torch.from_numpy(val_X.values).long()\n",
        "test_X = torch.from_numpy(test_X.values).long()\n",
        "\n",
        "train_y = torch.from_numpy(train_y).long()\n",
        "val_y = torch.from_numpy(val_y).long()\n",
        "test_y = torch.from_numpy(test_y).long()\n",
        "# 设置dataset\n",
        "train_set = Data.TensorDataset(train_X, train_y)\n",
        "val_set = Data.TensorDataset(val_X, val_y)\n",
        "test_set = Data.TensorDataset(test_X, test_y)\n",
        "# 设置数据集加载器，用于模型训练，按批次输入数据\n",
        "train_loader = Data.DataLoader(dataset=train_set,\n",
        "                               batch_size=32,\n",
        "                               shuffle=True)\n",
        "val_loader = Data.DataLoader(dataset=val_set,\n",
        "                             batch_size=32,\n",
        "                             shuffle=False)\n",
        "test_loader = Data.DataLoader(dataset=test_set,\n",
        "                             batch_size=32,\n",
        "                             shuffle=False)"
      ],
      "metadata": {
        "id": "1D-WS31iIO_8"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "*斜体文本*# 六.模型训练"
      ],
      "metadata": {
        "id": "b2ZtewcyvImT"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "#### 模型训练的一般步骤：\n",
        "#### 1.定义损失函数\n",
        "#### 2.定义优化器\n",
        "#### 3.定义模型参数可更新\n",
        "#### 4.遍历数据集训练模型\n",
        "#####  $\\qquad$*4.1输入数据，获得预测结果\n",
        "#####  $\\qquad$*4.2计算损失\n",
        "#####  $\\qquad$*4.3反向传播\n",
        "#####  $\\qquad$*4.4参数更新"
      ],
      "metadata": {
        "id": "XSCS29mbvQcI"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "def train(model):\n",
        "    # 1.设置迭代次数训练模型\n",
        "    for epoch in range(epoches):\n",
        "        train_loss = []\n",
        "        # 1.1设置二分类交叉熵损失函数\n",
        "        criterion = nn.BCELoss()\n",
        "        # 1.2设置adam优化器\n",
        "        optimizer = optim.Adam(model.parameters(), lr = 0.001)\n",
        "        # 1.3设置模型训练，此时模型参数可以更新\n",
        "        model.train()\n",
        "        # 1.4遍历训练数据集，获取每个梯度的大小，输入输出\n",
        "        for batch, (x, y) in enumerate(train_loader):\n",
        "            # 1.4.1如果有gpu则把数据放入显存中计算，没有的话用cpu计算\n",
        "            x=x.to(device)\n",
        "            y=y.to(device)\n",
        "            # 1.4.2数据输入模型\n",
        "            pred = model(x)\n",
        "            # 1.4.3计算损失\n",
        "            loss = criterion(pred, y.float().detach())\n",
        "            # 1.4.4优化器梯度清空\n",
        "            optimizer.zero_grad()\n",
        "            # 1.4.5方向传播，计算梯度\n",
        "            loss.backward()\n",
        "            # 1.4.6优化器迭代模型参数\n",
        "            optimizer.step()\n",
        "            # 1.4.7记录模型损失数据\n",
        "            train_loss.append(loss.item())\n",
        "        # 1.5模型固化，不修改梯度\n",
        "        model.eval()\n",
        "        val_loss = []\n",
        "        prediction = []\n",
        "        y_true = []\n",
        "        with torch.no_grad():\n",
        "          # 1.6遍历验证数据集，获取每个梯度的大小，输入输出\n",
        "          for batch, (x, y) in enumerate(val_loader):\n",
        "              # 1.6.1如果有gpu则把数据放入显存中计算，没有的话用cpu计算\n",
        "              x=x.to(device)\n",
        "              y=y.to(device)\n",
        "              # 1.6.2模型预测输入\n",
        "              pred = model(x)\n",
        "              # 1.6.3计算损失函数\n",
        "              loss = criterion(pred, y.float().detach())\n",
        "              val_loss.append(loss.item())\n",
        "              prediction.extend(pred.tolist())\n",
        "              y_true.extend(y.tolist())\n",
        "        # 1.7计算auc得分\n",
        "        val_auc = roc_auc_score(y_true=y_true, y_score=prediction)\n",
        "        # 1.8输出模型训练效果\n",
        "        print (\"EPOCH %s train loss : %.5f   validation loss : %.5f   validation auc is %.5f\" % (epoch, np.mean(train_loss), np.mean(val_loss), val_auc))        \n",
        "    return train_loss, val_loss, val_auc"
      ],
      "metadata": {
        "id": "NGS7BJyEXnq-"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# 定义din模型\n",
        "model = DeepInterestNet(feature_dim=fields, embed_dim=8, mlp_dims=[64,32], dropout=0.2).to(device)\n",
        "# 迭代次数\n",
        "epoches = 5\n",
        "# 模型训练\n",
        "_ = train(model)"
      ],
      "metadata": {
        "id": "6rcPq16AYmGu"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [],
      "metadata": {
        "id": "89ZKl4nAYn5u"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [],
      "metadata": {
        "id": "aLK9xxMVYplG",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "8408d320-ada4-4a30-c01f-a735f6ad6677"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "EPOCH 0 train loss : 0.69177   validation loss : 0.68750   validation auc is 0.54773\n",
            "EPOCH 1 train loss : 0.68296   validation loss : 0.67819   validation auc is 0.58642\n",
            "EPOCH 2 train loss : 0.67711   validation loss : 0.67410   validation auc is 0.59692\n",
            "EPOCH 3 train loss : 0.67433   validation loss : 0.67409   validation auc is 0.60121\n",
            "EPOCH 4 train loss : 0.67315   validation loss : 0.67269   validation auc is 0.59867\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# 7.效果展示"
      ],
      "metadata": {
        "id": "QmhmNaIDvEoE"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "#### 最后拿一条数据看下效果"
      ],
      "metadata": {
        "id": "2ndTjdnNw7kc"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "#输入的数据\n",
        "dis_test_x.apply(cate_encoder.inverse_transform).reset_index().head(1)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 174
        },
        "id": "6Ly0CVjApJ4J",
        "outputId": "c5b31a59-cd91-473e-c7b7-6b9d4d8ed4bb"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "   index hist_cate_0 hist_cate_1 hist_cate_2 hist_cate_3 hist_cate_4  \\\n",
              "0  53523       Books       Books       Books       Books           0   \n",
              "\n",
              "  hist_cate_5 hist_cate_6 hist_cate_7 hist_cate_8  ... hist_cate_31  \\\n",
              "0           0           0           0           0  ...            0   \n",
              "\n",
              "  hist_cate_32 hist_cate_33 hist_cate_34 hist_cate_35 hist_cate_36  \\\n",
              "0            0            0            0            0            0   \n",
              "\n",
              "  hist_cate_37 hist_cate_38 hist_cate_39 cateID  \n",
              "0            0            0            0  Books  \n",
              "\n",
              "[1 rows x 42 columns]"
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-de31b53e-6f87-4357-8f36-7a99c8ffe48d\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>index</th>\n",
              "      <th>hist_cate_0</th>\n",
              "      <th>hist_cate_1</th>\n",
              "      <th>hist_cate_2</th>\n",
              "      <th>hist_cate_3</th>\n",
              "      <th>hist_cate_4</th>\n",
              "      <th>hist_cate_5</th>\n",
              "      <th>hist_cate_6</th>\n",
              "      <th>hist_cate_7</th>\n",
              "      <th>hist_cate_8</th>\n",
              "      <th>...</th>\n",
              "      <th>hist_cate_31</th>\n",
              "      <th>hist_cate_32</th>\n",
              "      <th>hist_cate_33</th>\n",
              "      <th>hist_cate_34</th>\n",
              "      <th>hist_cate_35</th>\n",
              "      <th>hist_cate_36</th>\n",
              "      <th>hist_cate_37</th>\n",
              "      <th>hist_cate_38</th>\n",
              "      <th>hist_cate_39</th>\n",
              "      <th>cateID</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>53523</td>\n",
              "      <td>Books</td>\n",
              "      <td>Books</td>\n",
              "      <td>Books</td>\n",
              "      <td>Books</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>...</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>Books</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "<p>1 rows × 42 columns</p>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-de31b53e-6f87-4357-8f36-7a99c8ffe48d')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "      \n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-de31b53e-6f87-4357-8f36-7a99c8ffe48d button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-de31b53e-6f87-4357-8f36-7a99c8ffe48d');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n",
              "  "
            ]
          },
          "metadata": {},
          "execution_count": 69
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "#模型输入的向量\n",
        "test_X[0]"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "1ySOYc5-utzl",
        "outputId": "1de7bc03-cac0-4c5a-c2cc-3b51165fff28"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor([138, 138, 138, 138,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
              "          0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
              "          0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0, 138])"
            ]
          },
          "metadata": {},
          "execution_count": 70
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "#预测购买的概率\n",
        "model(torch.unsqueeze(test_X[0],0))"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "DEJVDINxqAax",
        "outputId": "31af8a74-dea1-41ac-a388-2c4266b42242"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor([0.5218], grad_fn=<SigmoidBackward0>)"
            ]
          },
          "metadata": {},
          "execution_count": 71
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "#事实上该用户是否购买\n",
        "test_y[0]"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "VNIJSIdGqfjJ",
        "outputId": "3a2488df-9553-4c12-b3da-a71680dc09ea"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor(1)"
            ]
          },
          "metadata": {},
          "execution_count": 72
        }
      ]
    }
  ]
}